The field of the present invention is product discrimination systems based on sensed electromagnetic spectra.
Product handling activities for the discrimination of product units have long been used, particularly in the food product industry. Such discrimination has been based on size, ripeness, color, blemishes and the like. Until recent times, this activity was generally undertaken by manual labor. The versatility of workers for handling and processing large amounts and varieties of food products has generally been unsurpassed. Such processing systems typically include a conveyor passing working stations where workers distinguish and separate product units. Difficulties in finding experienced seasonal workers and the normal administrative problems associated with a fluctuating work force have long created a need for less labor intensive systems.
In defining the needs for product handling systems, as particularly applied to the food industry, the nature, volume, relative unit cost and variety of products severely impact the design of handling equipment. Most food products must be handled with great care to avoid damage. The perishable nature and large batch quantities of products in season make rapid processing a necessity. The variety of products which must be processed at different times to economically justify a food processing facility places great demand for versatility on the equipment. Thus, a substantial challenge exists in creating handling equipment to replace the versatile human worker.
An early system for handling products in a manner acceptable for automatic sorting is disclosed in U.S. Pat. No. 4,106,628 to Warkentin et al. for SORTER FOR FRUIT AND THE LIKE, the disclosure of which is incorporated herein by reference. In this patented device, cups are arranged on a chain conveyor for holding individual product units. Solenoids act to dump selected cups for product separation responsive to discriminating sensing and electronic commands. Other separating systems include devices for batting or blowing selected units from a conveyor.
In the early system of U.S. Pat. No. 4,106,628, color from a product unit is directed through lenses, fiber optics and filters to a sensing mechanism. In the actual system, light from both sides of a product unit was gathered in a single scan per product unit by two bundles of optic fibers looking from opposed sides of the product unit. Each optic fiber bundle was split and combined with a respective split portion of the other bundle. Therefore, each resulting optic fiber bundle had light from both sides of the product unit. Filters of different wavelength capacity were employed to filter the light derived from the resulting two fiber optic bundles. Red and green filters were given as examples, one filter for each resulting bundle. The signals generated by the filtered light were then compared with a standard such that a red/green color classification could be made based on the readings compared with the standard.
Another system which has been in use for some time is disclosed in U.S. Pat. No. 4,961,489 to Warkentin for PRODUCT HANDLING SYSTEM, the disclosure of which is incorporated herein by reference. In this device, a conveyor is employed which includes elements capable of tipping to off-load individual units of a product being processed. The nature of the conveyor permits some variety in shapes and sizes, including elongated products. However, a range of round or oval products in smaller sizes is not as easily accommodated by this system.
Other systems which have been in use successfully for some time are disclosed in U.S. Pat. No. 5,029,692 to Warkentin for OFF-LOADING CONVEYING SYSTEM, U.S. Pat. No. 5,195,628 to Warkentin for OFF-LOADING CONVEYING SYSTEM, U.S. Pat. No. 5,215,179 to Warkentin for OFF-LOADING CONVEYING SYSTEM, and U.S. Pat. No. 5,474,167 to Warkentin for OFF-LOADING CONVEYING SYSTEM, the disclosures of which are incorporated herein by reference. A newer device is presented in U.S. patent application Ser. No. 08/800,602, filed Feb. 19, 1997, to Warkentin for CONVEYOR, the disclosure of which is incorporated herein by reference. In these devices, a conveyor is employed which includes concave rollers defining concavities therebetween. off-loading elements are positioned between adjacent rollers in each of the concavities. The elements are pivotally mounted about pivot axes and can be actuated to off-load product units.
A product discrimination system employing the sensing of a variety of light spectra, which may include wavelengths both in and beyond the visible spectrum, from product units being classified is disclosed in U.S. Pat. No. 5,018,864 and U.S. Pat. No. 5,106,195, both to Richert for PRODUCT DISCRIMINATION SYSTEM AND METHOD THEREFOR, and U.S. Pat. No. 5,223,917 to Richert for PRODUCT DISCRIMINATION SYSTEM, the disclosures of which are incorporated herein by reference. The system has particular utility in sorting food products such as fruits and vegetables. The magnitudes of the sensed light spectra is analyzed to determine such attributes of a product as size, ripeness, blemishes and color. A manageable amount of data is received and processed by such a system with a maximum number of product factors being determined.
In the system of U.S. Pat. Nos. 5,018,864, 5,106,195 and 5,223,917, a focused image of a product unit is directed to a fiber optic array. The array has a first end which is arranged in a rectangle. Because of this arrangement, the fiber optic cable receives what approximates a line scan image. The image may be averaged and then divided and directed through filters to provide a plurality of sensed signals for different wavelengths. Intensity may be measured for each selected wavelength spectrum. Consequently, only a few signals, the magnitude of each separately filtered portion of the image, need be processed. Methods for discriminating attributes of product units use absolute magnitudes and comparative relationships between magnitudes of various spectra of light sensed from a product unit to determine such attributes as size, color, ripeness and blemishes. Such methods may be carried out on a variety of sensing hardware including line scan cameras as well as the fiber optic system. Even a combination of such systems was suggested.
A system used with such off-loading conveyors for discriminating product units is disclosed in U.S. Pat. No. 5,156,278 to Aaron and Richert for PRODUCT DISCRIMINATION SYSTEM AND METHOD THEREFOR, the disclosure of which is incorporated herein by reference. A fiber optic system is used to sense physical attributes of the conveyed product units. A system for rotating the fruit between sensors provided added accuracy. Multiple sensing of the product is accomplished in series with a partial rotation of the product unit between each sensing and with the product stationary during each sensing. The rotation is accomplished by driving the supporting elements on the conveyor. Such rotation and multiple sensing provides substantial capabilities in the accuracy and variety of measurements derived from the process. An extended drive is provided for rotation of the supporting elements and, in turn, the product units on the conveyor prior to the sensing operation. Fruit and vegetable product units tend to be nonuniform and difficult to singulate and properly position on a conveyor. The rotation of such product units on the supporting elements tends to allow them to properly orientate, seat in a conveyor cavity and separate one from another such that sensing is enhanced.
More complicated sensing devices have been developed which use line scan cameras for determining such attributes as cross-sectional area. Such cameras have used light to present pixel information which may then be processed for summation and the like. For example, cross-sectional area may be determined by counting the number of pixels registering presence of the product unit. Such systems collect information in the form of pixel location, color or colors, and intensity, generating a substantial amount of data to be received and processed.
In a refinement of the system of U.S. Pat. No. 5,156,278 and using the features thereof, another system used with such off-loading conveyors for discriminating product units is disclosed in U.S. Pat. No. 5,286,980 to Richert for PRODUCT DISCRIMINATION SYSTEM AND METHOD THEREFOR and U.S. Pat. No. 5,401,954 to Richert for PRODUCT RIPENESS DISCRIMINATION SYSTEM AND METHOD THEREFOR WITH AREA MEASUREMENT, the disclosures of which are incorporated herein by reference. Both a fiber optic system and a line scan camera are used to sense physical attributes of the conveyed product units and the line scan camera is used for continued calibration. The two sensing systems act to measure area among attributes such as color. A comparison of the measurements for area using the two systems with the fiber optic system operating in a spectral range empirically determined to vary from the line scan camera as a function of an attribute of ripeness allows discrimination based on that attribute. The line scan camera also recalibrates on an ongoing basis by measuring a standard object and comparing the result with a standard value. Variations in products, conditions and sorting criteria can require experienced skill in changing applied spectral ranges. Also, defects are sometimes difficult to recognize because of the load of information and the sensitivity limits of the sensing systems.
The present invention is directed to a method and to apparatus for the discrimination of product units using electromagnetic spectra. The electromagnetic spectra is separated into spectral ranges for processing.
In a first separate aspect of the present invention, an image of a thin portion of a product unit is passed through a slit and a prism to spread the spectrum of the image perpendicularly to the major dimension of the slit and then to a CCD camera. An array can be generated for sensing by the camera which has spatial pixels distributed in a first direction and the spectra for each spatial pixel distributed in a direction perpendicular to the first direction.
In a second separate aspect of the present invention, the foregoing aspect may further include the treatment of the entire product unit sequentially as the unit is passed by the sensing system. This product unit may more specifically be a unit of produce. Attributes of the entire unit or portions thereof may be employed for discrimination using electromagnetic spectral images.
In a third separate aspect of the present invention, a number of techniques may be uniquely employed advantageously, either independently or in combination, to manage data with a system separating electromagnetic spectra into spectral ranges for processing. Binning of data in areas of system weakness can prove useful to overcome optic and camera sensitivity. Summing of pixel counts and pixel magnitudes in selected spectral ranges can be used to reduce data storage and manipulation. The creation of histograms also provides such an effect. Ongoing recalibration through reference to a standard can accommodate nonsteady state conditions such as changes in the light source and in system temperature.
In a fourth separate aspect of the present invention, a system separating electromagnetic spectra into spectral ranges converted to a digital format is employed with algorithms to measure and compare various attributes of the product units processed. Area and other attributes recognized by different spectral ranges can be used or combined to provide discrimination based on calculated size, maturity, color, shape and damage.
In a fifth separate aspect of the present invention, filter training is used in a system separating electromagnetic spectra into spectral ranges based on selecting product units with distinguishing attributes. Scanning of such units and processing the resulting image data with a pattern recognition routine can be used to select specific spectral ranges for use in product unit processing algorithms. Data magnitude can be reduced through such a process by 98% or more without significant loss to selected sensitivity.
In a sixth separate aspect of the present invention, algorithm training based on selecting product units with distinguishing attributes is used in a system which separates electromagnetic spectra of a line scan into spectral ranges. Scanning of such units with filtering in place and processing the filtered data with a pattern recognition routine can be used for generating feature estimation algorithms. Such a routine can be applied for initial setup, changes in the product processed, changes in the conditions of the product being processed or changes in selection criteria based on amended output requirements.
In a seventh separate aspect of the present invention, optics employing a slit and a prism are arranged in conjunction with a conveyor to produce an image array for digitizing by a CCD camera. The array reflects spatial location across the conveyor in a first direction and the wavelength of the transmitted electromagnetic radiation in a second direction perpendicular to the first direction. The image intensity of the array divided into pixels by the CCD camera may be the output.
In an eighth separate aspect of the present invention, the foregoing separate aspect may further include an objective lens and a relay lens placed to either side of the slit in optical alignment and with the exit pupil of the objective lens imaged onto the entrance pupil of the relay lens to better achieve the desired spreading of the relevant spectra. The transmission and sensing of a wide range of spectra is preferred. A range of 450 nm to 1150 nm is contemplated.
In a ninth separate aspect of the present invention, any of the foregoing aspects are contemplated to be combined for advantageous application.
Accordingly, it is an object of the present invention to provide improved discrimination of product units. Other and further objects and advantages will appear hereinafter.